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METHOD 



- zzp or-cn ne. -'5 en evcluation ruciel wh'ich facilitates informed 
decis, ■ j-.i' g. Tne jority o~ prograryis, studies or projects are evaluated 
becau' dec sfc^-n^c wants help in figuring out what to do and what choices, 
to cr-er to '2e out-Ories. 

■•• qLc: tion ze:. ^ne cf deciding whose outcorMes should be rr.ax i mi zed. 
Occi ■ IhGj' a ^- -.e • j^.cjmes for the deci s-jcn-maker as they are under- 

stC'Dc ihos. wh:) ser . anc -.idvise him. Kare often, some combination of the 
val'u'" ' " rr.any diff^-^t- . griups w;th interests in the decisions should be 
naxir :. 

evai^aticn p j*^, must C':rsist of a technology for extractir>g, reconcile 
expli ng ^.nd acjre'"-:ing the inconsistent values of various croujps in order 
that ial decisvon ai- ing -can . be undertaken. 

T e evaluation Ian that was proposed would assess progranri value, 
cort' nurusly taking ' nto account the values of those served by ".he program. 

Df^ta was gazhe- -^d that could be used as information to obtalii a set of 
sir:p'e tnswers ir, or " to make informed decisions. Each decision situation 
has a rjititude o*-' vc le dimensions impinging on it. The problem was to 
dete-^rr'ne the dim-nsv is that were relevant and how relevant each is. Other 
questions that needed :o be answered were: Who determines the relevancy? 
How are the judgments "ade or used? How is the location of each possible 
alternative on each c rnension of value measured, judged or found? What 
trans fc rmati ons or aggregation of input information is utilized to translate 
this i'put into outcofnie evaluation? 

li view of the experimental nature of the project and the need for 
inf onr^a '.ion as a base's for future policy decisions the major focus of the 
evaluation was of a formative nature. 

In order to accomplish this formative evaluation a great deal of fact 
gathering was indispt'isable. Fact gathering by itself suffers from two defi- 
cienc^'es as an evaluation method: 

1. Too many facts are ali-iost as difficult to use for making decisions as 
too few. 

2. Valuer almost incvitc*bly escape the fact gathering nu::.l^er dredge. 
This ^'s probably so because they reside in the deci si on-r^akers head 



[jV, rr.ore often, in the collective end disagreeing h^^- id -'"the 
various orgeni Zctions that have a sa^' in the decisior 
In. --his -^^-iign, fact gathering was used for two rr.ain purposes: 
1. To describe the project^ 

Td generate recoin^endations to improve the project. 
I' majority of decis.ions there are a multiplicity of vc 'i. dinensions 

thdt F ■• t a multiplicity of problems. Who determines what :-r'prsions are 
releva' i. -.nd how relevant each is? How is that set of judgme-^ts ' .ace or used? 
Hov: is :re location of each possible outcome of each act being cc* . icered on 
ea'::h r--; vant dimension' of value measured or judged or discovered? .Jhaz 
CO " .::.in„ t :on of judgmental, transformati on and arithmetical aggr=r^gction is used 
tc tra ;5 cte al'^ this input information into outcome evaluatic's" 

ie/eral versions of an explicit technology exist to av er seme of these 
c'jest' .f^s.. Its name is mul ti -attri bute utility measurement a : expositions 
of var cus versions of it have been presented by Raiffa (195£ , Keeney (1972), 
Edv;ar-: ( 1971 ) and others. 

"r.e version suggested here is adapted from Edwards (]9~1) and is oriented 
towar^- easy communication and use in environments in which t-me is short and 
decis n-nakers are numerous and busy. Further, it is a method that is 
psyche' ogical ly meaningful to decision-makers, who are required to give judgments 
thr' 'e "ntuitively reasonable, 

^le essence of mul ti-attribute 'uti 1 i ty measurements in any of its versions 
is thai each outcome to be evaluated is located on each dimension of value by 
a procedure consisting of experimentation, naturalistic observation, judgment 
or sone combination of these, A simple linear weighted combination rule will 
suffice to aggregate these dimensions and outcomes. The weights a)"e numbers 
describing the itr,portance of each dimension of value relative to the others. The 
numbers are judgmentally obtained from the participants cf the project. 
The implementation of the technology consisted of ten steps: 
Step 1: Identification of the persons or organizations whose utilities 
are to be maximized. People who can speak * or them must be 
identified and, induced to cooperate. 
Step 2: Identify the issue or issues (i.e. decisions) to which the 
utilities needed are relevant. The same recomiiendati ons may 
have many di f ferent. val ues depending upon their nature and 
purpose. 

Step 3: Meritify the entities to be evaluated. Often it is sufficient 

to treat an action itself as an outcome. This ai cunts to treating 



the action as having an inevitable outcor^ie, that is, of ass-uT^ing 
that 'jncertainty about outcorTies is not involved nn the evaluation 
of that action/ These 'entities' are the reqornriendati ons 
generated from the data analysis of questionnaireSi interviews, 
etc. 

Identify the. relevant dimensions of value. The first three 
steps are more or less philosophical. The first answered the 
(Question - Whose utility? The second answered the question - 
Utility for what purpose? The third answered the x^uestion - 
Utility of what entities? 

Step 4 asks us to discover what dimensions of value are important 
tc the evaluation of the entities we are interested in. 
It is important not to be too expansive at this stage. The 
number of relevant dimensicns of value should be kept down. 
Eight dimensions "are plenty and fifteen are too many. 
Rank the dimensions in order of importance. The ranking job can 
be performed by representatives of conflicting values acting 
separately, or by those representatives acting as a group. 
Rate dimensions in importance preserving ratios. 
^Sum the importance weights, divide each by the sum and multiply 
by 100. ■ This converts the importance weights into numbers that 
are similar to probabilities. 

Measure the location of each entity or recommendation that was 

generated from Step 3 on each dimension- The word measure is 

used loosely. There are three classes of dimensions, purely 

subjective, partly subjective and purely objective. The purely 

subjective dimensions are perhaps the easiest. Simply get an 

appropriate expert to estimate the position of that entity 

or recommendation on that dimensi.on on a 0 to 100 or 0 to 10 scale 

where 0 is defined as the minimum plausible value on that 

dimension and 100 is defined as the maximum plausible value. • 

This is probably the simplest way to proceed. 

Calculation of utilities for entities. The equation is V^- = 

tiK'j Vij remembering that^j Wj = ICQ, Vi is the aggregated utility 

for the ith entity on the jth dimension. Thus Wj s the output 

of Step 7 and Uij is the output of Step 8. 

Decide. If a single ^ct is to be chosen then the S'.lrset for which 
\^ \ is naxinum is best. 



Indopendpnce Properties 

The cogrecation rule assumed value 1 ndeperidence. ?^ough]y, that means 
tr.ctt the extent of your preference for locat on over location a^ on 
di.nensi'on A is unaffected by the position of che entity being evaluated on dir;ensions 
B, C, D. Fortunately, ^n the presence of even mociesi amounts of r.eas^rer.en: error, 
quite subst'^rre-l amounts of deviation froj^ vaTue independence will make little 
di f fapei^ee' to the ulti.r^ate number V^. , and even less to the rank ordering of the 
V- "values. (Da.wes and Corrigan (1974).} 

In order that entities or recorrriendations can be generated for Step 3, a 
variety of data col 1 ection' Jnsirurnents was utilized. .The majority of these can 
be found -in Evaluation Education, C.T.S. Number 1 evaluation model instrum.ents 
1975-77. 

The data was^coTlected by questionnaires, interviewing and anecdotal nctes.'' 
A" variety of stati s'tical analysis was undertaken on the 'data to generate the 
entities pr recornmenda lions required in Step 3. In addition recommer^dati ons * 
solicited by respondents by questionnaires was utilized. 



Six quest! cnnci res werp iit.iiized to collect data related to de^iogrdphic 
inforr-iiion on the respondents, especis of the siudio envi rcnr-;ent , anioL'nt of 
interaction, organization, quality of conrrjni cati on , satisfaction of technical 
aspects, interest of content and t:'ansmiss Ion conriiticns. In addition, the 
questiorinai res allowed for corninents and recorriTiendati ens . Copies of each of 
these cjesticnnaires and the results nay be fojnd ir Appendix A, 

Questi rTtr.ai res were distributed tcr studio locations at British Cclurr.bia 
Institute of Technology, Burnaby, Okanagan College, f:elowna, Fraser Valley 
College, Chi>iiwack, i>orth Island College, Carrpbell River, iNorthern Lights 
College, Dawson Creek, and Pitt LaKe. Participants at these sites were 
encouraged to fill out these questionnaires. Initially one type of program 
frorr; ec'Ch of ihe following areas was chosen to -::irveyed. These areas we'^e 
Forestry, Medical, Leg^K Perspectives, Psy ' .tellite College, 

Computer Data research and one general proc,' er as a consequence 

of low attendance for some of these programs, ani * -.^s at sites were requested 
to distribute questionnaires according to audience participation. It is 
probable that the responses were respresentati ve of the majority of programs. 
Although, had other programs in a cluster been surveyed one cou'^d expect 
some variations in responses. Representativeness was further insured by the 
selection of programs over a number (9f different days. 

ANALYSIS 

A breakdown by program type, e.g, for Medical, etc. was not carried out 
because these questionnaires related mainly to the technical aspects of the 
evaluation. In addition, the evaluations by the consortium members regarding 
their own program should adequately cover this aspect. 

Questionnaires were frequency tabulated. In additicm, crosstabul ations 
were carried out to determine the differences in responses by location site, 
Findlly, regression analysis was undertaken to provije information as to how 
some of the respondents were making their judgments regarding the communication 
aspects of the project. 



RESULTS 



This section presents the results of the analysis. 

FORM A . 

-D emographic Characteristics of the Respondents 

This form is concerned with the physical layout of the onsight location. 
There were 71 respondents to this form. The majority of the respondents were 
•20-25 years of age (43%). The second largest qroup consisted of 26-40 years 
of age which represented 35% of the respondents. The^ remaini ng 22% were over 
40 years of age. The educational background was approximately divided by 
50% secondary and 50% college or university. Of these respondents 67% were 
Male and 33% were Female. 70% of these respondents lived in a city or town, 
20% lived in the country and 7% lived, on farms. 

Studi 0 Envi ronment 

The actual layout of -the studio as .estimated by the respondents showed 
a range from excellent (40%), good (34%) to poor (22%). Approximately 90% 
of these respondents thought there were. enough T.V. monitors in their studio. 
This same percentage thought th^, aVraggemeht of-these monitors was satisfactory 
for viewing the program. The plctiJre^qual i ty of the mof;itors was excellent 
(67%) and the overall adequacy of the T.V^ picture for achieving the Teaching 
Learning objectives was considered to be good to adequate (80%). 92% of the 
respondents were of the opinion that there were enough speakers to hear' the 
programs. Whereas only 68% thought the sound quality to be in the range of - 
good to excellent. 29% thought the sound quality to be adequate. Approximately 
half of the respondents were able to hear everything clearly on the system. 
76% of the people believed the, sound system to be adequate to achieve the objective 
of the programs. 

The majority (86%) thought there were enough microphones for participation. 
Approximately one-half of the groups in the sessions surveyed used the 
microphope. About 82% were of the opinio^n that the procedures and techniques 
for equipment use were c-1 early explained. 'Of the people who used the equipment 



90^ found it easy to use and 82% had no problems using the equipment. Echos 
were a very minor problem, as only 18% experienced any difficulties with them. 
75% of the respondents believed the total communication system to be adequate 
for achieving the objective of the programs. 88% expressed the desire to use 
this type of communication system for similar programs. 

FORM 3 

Communication 

This form deals with the communications aspect as a two-way experience. 
There were 61 respondents to this form. Of the sessions surveyed the majority 
of the respondents indicated that either'] or 4 other locations were contacted 
by members of their group. However, there could have been a misunderstanding 
in the interpretation of this question because four sites communicated with 
the British Col umbi a I nsti tute of Technology (BCIT). The responses indicate 
that only a few of the people present at a location talked with other locations. 
In the majority of cases (64%) the session did not start pr ior \o sate! 1 i te 
communications. For the 36% of the respondents in which the session started 
earlier the majority of the time was spent explaining objectives, learning to 
operate the equipment and preparation of the content. 78% of the respondents 
stated there was a moderator or group leader on site. 26% believed that the 
program most resembled talk-.ing on the' telephone, 23% listening to an 'open line* 
radio show, 21% having a conversation with friends. 54% agreed it was easy 
to identify who was talking on the system and 16% disagreed. Two-thirds of the 
respondents believed participation was encouraged by the group. The same 
percentage did not think that the equipment was a distraction and 64% did not 
feel technical adjustment took a long time. 

The ' respondents were equally split on, whether or not the sessions followed 
a predetermined plan. 65% felt comfortable with the system while 20% felt 
self-conscious. Approximately three-quarters of the people, found the 
telecommunication, system easy to use. It was found that the respondents 
differed in their opinion regarding aspects of closeness in gdl^graphical distance, 
57% agreed that the sites seemed close to each other while 33% did not. 59% ' 
of all the respondents ragreed that Satellite teaching was as understandable 
as "race-to-face*' presentation. 

During the-^course of the sessions 44% felt that questions and comments 
alteredsthe program^ and 37% disagreed. 67% of the respondents fel t the sessions 
were not long enough-. The respondents were also divided in thei r opi nion on 
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whether or not they got a good idea of how participants were reacting at other 
sites 59% vs, 47%. Only 15% of the respondents thought tfiat a disporportionate 
amount of time was spent on administration and operational procedures. About 
one-half of the respondents got a real feeling of personal contact. 15% thought 

• content of the sessions was presented in a confusing manner whereas 6^% did not. 
64% also felt that content of the sessions was interesting. 

The majority (84%) felt that the two-way capability vas essential for 
their sessions. 72% felt that the video image was essential. 48% felt that it 
was difficult to talk to people in one's own group during the satellite session 
while 37% felt that it was not and about 50% had discussion occur that was not 
transmitted to other sites during the session. 72% of the respondents strongly 
disagreed that during the sessions satellite contact had not been lost while 
18% felt that it had been lost. Most two-way communi cation which occurred 
on the system during these sessions appeared not to be carefully planned (54%). 

The^majority (82%) of the person(s) presenting the major portion of the 
content for these sessions appeared to the audience to be .comfortable using- . 
this type of system. Also 50% of the 'audience found that the spontaneity found 
in more conventional classes. was not inhibited using this satellite system. 
The majority (78%) found it exciting to be part of this experiment. The 
respondents (88%) felt thai: the general' level and quality of communication in 

'these sessions was highly acceptable to acceptable. 

FORM 2 ■ ^■ 

Technicians Log 

This form deals with the weather and transmi ssi ojn conditions at each site 
during the broadcasts of the programs. There were 15 respondents to this form. 
75% of the satellite project was broadcast with cloudy and wet conditions. 
The video transmission and reception during the sessionsvvas adequate and the 
audio transmission and reception was also adequate, The overall operation of 
the system during the session was>adequate. Dawson Creek and Pitt Lake on several- 
occasions had equipment failures and BCIT had a breakdown of the DOC 
transmitter on December 13, 1977. 



FORM 3 

Opinions and Attitudes 

The objective o- this questionnaire was to record the opinions and attitudes 
towards the satellite sessions. There were 68 respondents to this form. The 
pverall opinion of the relationship betweer one site and another site during the 
sessions revealed that they were unequal (74%), competitive (73%), and unfriendly 
(62%). The actual content of the program produced negative feelings among the majority 
(66%) although 64% of the respondents felt that the programs were aimed at specific 
audiences. The overall opinion of the satellite project was good (60%). 65% of 
the respondents felt that the -objectives of the session were clear and about 30% 
felt that the objectives were not achieved at all. 42% felt that conversation was 
difficult under these .condi tions . 

FORM 4 

Demographic Characteristics of the Persons involved. in Project 

This form was completed by persons involved in the planning, implementation, 
operation- or evaluation of the Satellite Project. There were only 64 persons who 
responded to this questionnaire. The majority of these respondents . (64%) were under 
40 years of age. 69% were males and 31% were females. Educaticnc:! background showed' 
that 88% had College or University education. They were engaged, for the most part 
in^Teaching (20%), Administration (28%) , and. Technical (15%) jobs. 

Training and Participation in the STEP Project 

42% of the persons participating in the project stated they were participating 
for personal and/or professional interest. The other reasons ranged from requiring 
- to participate by supervisors to substitution for another person, each of which was 
below 13%. The majority (84%) of these people had used audio visual equipment 
before becoming involved in this project. About :ne-half had worked with telecommun- 
ications equipment prior to the project. Two-thi/ds of the respondents were of the 
opinion that they had received enough training for the project, one-third did not. 

Support: for Project 

80% of the persons involved. in the project received support from their 
institutions Audio-Visual Department and "0% did not. 89% received support from their 
colleagues while 11% did not. Interestingly, only 45%Veceived support from 
departmental Chairman, Dean and Vice-Presidents of their institutions and 55% did 
not. 47% got help from the GTS evaluation team and 21% received help from their 
Uni versi ty> Commi ttees . . " ^ 

ERIC ■ ., la 
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Evaluation Criterion 

The majority of the respondents, about 34 , believed that the most imporian' 
criterion for measuring the success jf the project was the relevance of the project 
to the institutes objective ^nd/or mandate. The next largest group (25%) believe... 
that success should be measured in terms of innovation in Teaching and Learning. 
The next largest group (205^) believed it should be based on the satisfaction of the 
student. Only 2% believed it should be based on the satisfaction of the e^xperimenters . 
3^ on possibility of contini/ing the project. 

. Work load and Associated Factors 

53% responded that they had 1-5 departmental colleagues associated with the 
project. If we assume an:average of 3 colleagues per respondee, this group would 
have approximately 100 people, involved, 19% had anywhere from 16-20 colleagues 
involved.. This latter group would represent approximately 342 people if we take an 
average of 18 colleagues per " respondents , 3 people or approximately 5% had more than 
20 people involved in the project. This group, would represent ^ minimum of 60 
participants.' Includi-ng all respondent and individual associates with this project 
approximately 530 individuals participated on the basis of these responses. 'However, 
since the respondent rate for this form was only in the order of 50% it is quite 
probable that approximately 1,000 individual participated in this project. However, 
there may have been some possibility of double counting in' obtaining this estimate. 

With regard to the amount of time spent on the project about 58% of the respon- 
dents spent one-quarter of their time or less on this work, between one-quarter 
and one-half 13%, and 13% between one-half and three-quarters , and finally 11% between 
three-quarters and all of their work time. The majority (77%) of those involved ' ■ 
in the project conducted their regular duties along with the STEP project and about 
13% obtained some help. 36% estimated that their involvment in the project increased 
their work load by 25%,. 11% were increased by 50% and 5% were increased by ^5%. 
At>out three-quarters of the persons involved in this project were of the op'nion 
that it was mostly a team effort. Approximately 95% were of the opinion zhaz the 
project met or addressed a real need while only 5% did not. 64% believed that the 
project will be able to continue by using other resources and 36% did not. 

Satisfaction . 

64% of the respondents were very satisfied with their part in the project. 
Only 10% were dissatisfied. 
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^J123L9]ll ° cati on Site - Briti sh Colu nibia Ins ti tute of Technology 

Demoo raphic Characteristics of the Respondents 

This form was made up especially for the live studio and ?LPOst every program 
was evaluated. There were 50 respondents to this form. The majority of the respondents 
at this site (5?^) were 31-50 years of age and the second largest group were 20-30 
(46^'). The educational background was approximately divided evenly between people 
who had secondary education and University or College education. Of these respondents 
S2% were female and 48% were male. 90% lived in a city or town while the remaining 
percentage lived in the country. 

Studio Environment 

The actual layout of the studio showed 46% thought it was excellent, 36% good, 
and ]Z% poor. 70/c of the audience" felt there were enough TV monitors for the size 
of the group. The arrangement of the TV monitors was excellent (54%), good (2^2%) , 
and poor (24%). The picture quality of the moni tors >/as excel lent (86%) and the 
adequacy of the picture for the teaching-learning objectives was also excellent (70%)I 

There were enough speakers f9r the group .(92%) but the quality of the sound 
was only rated good by 68^. The overall adequacy of the sound system for achieving 
the teachirvg-learning objectives rated 58% adequate, 30% good, and 12% inadequate. 
The overall adequacy of the communications system for achieving the teaching-learning 
objectives was 70% adequate, 16% good, and 8% poor. 78% of the respondents felt 
that this type of communi ca ti ons system could be used regul arly for taking similar 
courses. 

The situations most resembling the impressions of the sessions at this site 
showed 38% thoiught it was like doing a 'live' TV broadcast, 22% thoug,ht it was • 
like, taking part in a. seminar., 14% thrqught it was like listening to an 'open line* 
radio show. 

In spite of geographical .distance the studio respondents (80%^3felt that the 
di fferent groups seemed relatively close to each other and 78% could easily tell 
what other site was talking on the system. This was primarily due to the fact that 
the user usually stated who was cal 1 ing and also the panel animator would direct 
the question time to the different sites by identifying the site before the question 
•..j:. 'aired. 

66% of the respondents felt the content of the programs was interesting and 
80% felt that the program f oil owed a 'predetermi ned plan. 57% disagreed that the 
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sessions were too long. According to 46%, most two-v/ay communication which 
occurred on the system during the session did not appear to be carefully planned 
while only 36% thought it was carefully planned. The overall opinion of the 
general level and quality of communication in this project was highly acceptable 
(82%) ^nd acceptable (12%). 

COnMENTS 

»■ « * 

Comments by respondents to questionnaires are presented in App<'"ndix B. 

RECOMMENDATIONS 

As a consequence of the analysis of both data and comments, meetings and 
anecdotal evidence, ".a list of recomiiendati ons were generated. These recommendati 
were divided into four main categories. 

1. Production 

2. Program Content 

3. Communication 

4. Audience 

Forty-three recommendations were gen5rated. These recommendations may be 
found in Appendix C. 

OBJECTIVES 

As a result of a meeting with the head of the D.E.P.G. objectives for the 

experiment 'were formulated, rank ordered and priorized. A summary of th^se 
objectives may be found in Appendix D. 

RE SULTS OF UTILITY ASSESSMENT OF RECOMMENDATIONS 

This section presents the results determi>ned by applying Step 8 of the 
method section. Essentially the utility of a recommendation compared to every 
other recommendation is assessed. For reasons of simplicity of presentation, 
a simp le 1 inear transformation was applied to the values calculated in Step 8. 
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In essence this involved takirig the highest nuniber j- '.led by Step 8 of the 
method, equating it to TOOO anc re-va1i"ing the rema g numbers accordingly. 
These resuUs are presented in Appendix E. 
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DISCUSSION 

r.rcommendd ti en #9, *More leac time for program prepar - lion and planning 
De given.*, shows three different utility valu'es, on extremely high, 
e in jpper-*med-i urn range, and one in the upper-lower range. This recommendation 
. n be seen to have extremely high jtility with respect to :.he co-ordinating 
function 0^ a centralized DE Agency Apparently 'this reconi; -^ndati on is c(|nsidered 
highly essential to adequately prepare for the planning and preparation of 
programs. It is interesting to not*: tnat for this project lead time appeared 
to be inadequate. For future programs, adequate lead time to co-ordinate 
ic'js agencies should be allowed for. 

This same recom-nendation has also relatively high utility with respect 
tc testing the feas i bi 1 i ty of varioi;s education agencies producticn -and shilling 
programs for general or specific audiences using a variety of modes. Ft is 
important ihat in future enough lead time is given to allow these agencies to 
interact with each other, identify tneir audiences and explore the best ways 
of prese^nt-ing information to these audiences. The recommendation also had 
a low degree of utility with respect to assessing the test cormunity needs. 
This ijiaybe duetto the fact thjt the decision makers in this project see a 
needs study connected to lead tif?ie as separate from the project. 

-With respect to recommendations 35, 40, 41, all of which deal with facets 
of participation or interaction, itc^r be seen that these activities have a 
very nigh utility. .These recommendat ms should be carried out if it is desired 
to produce programs that allc^ for interactive cGTimuni cation between students 
at a distance and a centrally located instructor. These recommendations nave 
only medium utility with respect to testing configurations of OE satellite 
based systems. In tnis project the responses indicated that a great deal of 
interaction took pla-- , The nature of this interaction was mostly between the 
studio location and one or more sites. Only in specific programs was their a 
great deal of onsight to onsight interaction. These results would suggest that 
future programs might consider more controlled interaction among sites rather 
than through a central studio, ^ , ' 

Recommendation #17, 'Needs study required to determine program content*, 
had a very high utility which indicates that a ;;eeds Study should be undertaken 



to de* -nrine program content. This recommendation was rated highly to 
aitair Objective #3 and Objective ttII. With regards to this project a 
Needs Study was not carried out. As a result there was dissatisfaction with 
the program content produced by some of the educational agencies. It 
seemed apparent that some of the better pre rams, in the opinion of the 
audience, were these in which the audience was identified, e.g. medical 
pro^ram, forestry. In addition, the main education program on how to 
teach librarians to use compute'^ terminals for library searches was especially 
successful. 

In conducting future Need Studies it seems important to identify 
specific programs and match these with -specific audiences. In addition, 
the location of the audience should be considered. This is ..ecessary 
" because towns such as Dawson Creek- do not have a cable system, therefore 
limiting the beaming of programs to specific on-site locations thereby , 
restricting the audience. 

Recomir ?nda t i ons #24 and #25, 'Co-operation from higher administrative 
leve s shou'-i be ciKOuraged* and ' Consort! um' members pruJucing programs - 
should adequately budget for their programs*, received high utility ratings 
but .-ere not carried out to any great degree. The consorti um members 
indicated by -their responses tha.t they had obtained very little supportive 
assistance from the higher administrative levels of their institutions. It. 
is reronmended thaz steps be taken to open lines of communication with the 
admir is::rati ve sta^f of the institutions. 

Recommendation r32, 'Animators may .wish to inform and prepare 
them^^lv'es more adequately regarding the content of programs* .was very 
usefti ut again was only carried out to^a limited extent^ in the project. 
It is recommended that' more attention be paid to this aspect. Although 
there v.^s quite a bit of publicity in the ijiitial stages pILjthe project and 
recor-nendati on ^36 (Increased publicity to encourage public participation is 
required) was judged to have high utility, some form of continued pub! icity 
might be considered, especially toward target audiences. 

Recommendation #3 (More time should be allowed for interaction and 
question periods. Time limits for panelists responses should be set) and^ 
recommendation #23 (More visual aides, e.g. slides, graphs, etc., should 
be 'uti 1 ized duri ng lectures) concerned technical aspects of programming 
and had high utility but again were not carried out to any great degree. 
Consideration should be given to carrying out these reconinendations more 
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fully. R-comrnendations r30 and r33 also deal with technical aspects of 
prograniTiii g and these should also be given consideration. Recommendations 
i^l, 5, 12, 13, 21 and 22 all deal with aspects of programriing and panelists 
should be carried' out more fully than .they were in the series of programs. 
.RecoiTimendations nl5, 35, 41 and 43 deal with participation and interaction 
as well as placement of monitors and again can be carried out to a greater 
degree than they have been in thesQ programs. 

It is important that if on-site locations are going to be used, that 
physical layout of these locations be adequate. Two-way video may wish to 
be utilized only in h*^'ghly technical circumstances. 

In view of the fact that the use of the Satellite for teleconferencing 
would be a cheaper, more efficient and suitable form of teleconferencing, 
especially for officials separated by great distances, more of this type of 
activity should be carried out. With respect to this activity Recomnendation 
=16 (Ways should be explored to assure confidentiality on teleconferencing) 
is crucial. Perhaps the problem of confidentiality in this type of 
telecoi^jrencing can be overcome with the use of suitable scrambling devices. 

Tne above discussion has outlined a few of the major problems encountered 
during these programs.^ It is hoped that some of th^se recommendations be 
given .attention in order that improvements can be made. 

In view of the rtumber pf ;pommuni ti es in the interior that have cable - 
systems and those that do not, serious considerations should be given to 
providing educa ti onal ^programs that would service both of thes type of 
communities. With regard to the content of these educational programs, 
sharing of pre-packaged programs from a variety of sources e.g. Europe, 
United States, amy be advisable. In addition, a specific program or course 
may be cr eated and pa ckaged by the ^-^^T^trv hn-a£Sf^f^nr\ a recfprocaT 

— x 

basis. . — 

Of great value would be the use of the SAtellite to link provincial 
college libraries with data banks in the lower mainland and the United States, 

The use of two-way audio could be restricted to highly specialized \ 
programs that may require these kinds of facilities e.g. teleconferencing 
and site to site* interactions. t 
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ERIC 
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QUFSIIOf^UAIRES 
' I NSTRUMENT 
FORM A 



No. of respondents from each site* 

BCIT 9 

Chilliwack 32 

Kelowna 25 . 

Dawson Creek 5 - . 

Program evaluated: No. of respondents per program: 

Forestry 20 

Medical 8 

Legal 9 

Perspective 8 

University of Victoria 10 

Satel 1 i te Col 1 ege 16 

FORM B ' ' • 

No. of respondents' from each site: 

BCIT £ ' • 

Chilliwdok 24 
Kelowna 34 
put Lake 1 

A — — • 

Program evaluated; No. of res^ofttter.ts^per program: 



Forestry 

Medical 

Legal 

Perspecti ve 

University of ViQieria 
Satel 1 ite Col lege 

I NSTRUMENT #2 

Technicians log - 12 

. INSTRUMENT ^3 

No. of respondents from each site: 

BCIT 4 

Chilliwack 28 

Kelowna 31 

Pitt Lak. 5 



n 

12 
7 
9 
7 
9 



Program evaluated: 



No. of respondents per pronram: 



Forestry 18 

Medical . 8 

Legal . 11 

.Perspective ' 5 

Univers.ity, of Victoria 15 

Satellite 'College 11 

INSTRUMENT ^5 (Questionnaire for BCIT only - studio audience) 

No. of respondents: 50 

Program* ev.a.lua ted :• " No. of res^pondents per program: 

Oistance Education .Samp'.er - 9 

Forestry '14 ' 

Medical - 16 

Legal 7 

Computer -•On-Line Searching 4 



COMMUNICATIONS TECHNOLOGY SATELLITE 



Educational Experiences Evaluation 

Instrument r1 
(To be completed by participants) 
Part A 

Introduction 

The prdject which you are participating in is one of a series of 
experiments examining uses of the Communications Technology Satellite 
for Education. 

To help us better understand t],^ potential and limitations of 
satellites for education, we are asking selected participants in the 
educational projects for their impressions and opinions about their 
experiences when using the satellite. 

Today we are asking you to answer questions about some technical 

aspects of the session you just completed. Yoiir answers can help in 

the design of components for future satellite communications systems. 
Please answer carefully. 

Instructions : 

Complete each of the following questions by selecting the most 
appropriate answer or filling in the information, requested . 

Don't answer questions which do not apply to today's satellite 
session. • ^' 

NOTE: We a**e asking for your name in order to group this questionnaire 
with others which you may 'answer during the satellite sessions. - 
Although the results of your questionnaire will be used, your name 
will remain confidential. 



1 . Name : 



2. Age: Under 20 ( 18); 21-25 02 ); 26-30 ( 5 ); 31-35 05 ); 36-40 ( 6 ); 

41-45 ( 4 ); 46->50 ( 1 ); 51-55 ( 6 ); 56-60 ( 2 ) ; ^1 -65 ( 1 ) ; 
66r0ver ( 1 ). . • . 

3.. Sex: Female (21 ) Male (50) . 

4. Educational background (specify in terms of highest level completed) 

elementary ^5); Secondary (28); College Ol ); 

University - Bachelors (14); Kasters (9); PhD (2),. M,D. ( 1) 

5. Where is your home? > • . . 

In a city or town ( ; In the country, but not on a farm 0^ )i 
In a village ( ^ ); In the country on a farm ( 5 ). 



- 23 - 



6. Date of this satellite session. 

Day ■ Month Year 

SCALE TO REPRESENT YOUR OPINION OR REACTION. 

7. How would you rate the physical layout of the room used for this 
session? 

Excellent A. ^ ^ li- J_ ^^^^ 

8. Were there enough TV monitors in the room for the size of group 
v;6tching the sessnon? 

Yes 60_ J_ J_ J_ _2_ No 

9. Were the TV monitors arranged so that you could satisfactorily 
see what was happening? 

Yes 58_ A- A, JL J_ No 

10. ' How good was the general quality of the TV picture for this session? 

Excellent 32_ 1_6_ 1J_ _8_ __4_ Poor 

picture quality' picture quality 

11. Rate the overall adqueacy of the TV picture for achieving the 
teaching-learning objectives of this session. 

Adequate 35_ 22_ 10__ ^ _L- Inadequate 

12. VJere there enough loud speakers for everyone to satisfactorily hear 
what was being said? 

Yes 56_ J_ __4_ _2_ No 

13. How good was the sound quality? 

Excellent 22_ 26_ T6_ J_ J_ Poor sound 

sound quality quality 

14. How would you rate the volume of the sound? 

. . . Too loud 4 -.9 1 1 . Too low 

15. U'ere you alv/ays able to clearly hear everything that was said on the* 
system? 

Yes, always 23 12 9 1 3 14 No, not always 

16. -Rate the overall adequacy of the sound system for achieving the 

teaching-learning objectives of this session. 

Adequate 39 15 1 2 5 • Inadequate 
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17. Were there enough microphones to allow ready participation by any 
member of your group? 

Yes 52_ _Z. 

18. Did you use, a microphone during this session? 

Yes 34_ No 35_ . ' - / . 

19. Have the procedures and :techniques for using the equipment associated 
with this session been clearly explairred to y^ou? 

Yes 58 No n 7^^^ 



20 Do you find the equipment easy to use? 

Yes, easy 52 _2_ 3_ J No, difficult 

21. Were there any problems with the equipment in your room during this 
session? 

Yes, Many 5_ 5 3 . ^ 4q No, none 

22. When a member of your group v;as on the system, could you hear the 
echo of what was said coming from other .locations? . : * • ' 

Yes, always 5 7 3 9 44 

No, never 



23. Taking into account the teaching-learning objectives of this session, 
rate the overall adequacy of this communications system for ac^hieving 
them. 

Adequate ^9 ^] J Inadequate 

24. Would you like to regularly use this type of communication's system 
for taking similar courses? 

Yes 59 No 8 

25.. a. Do you have any suggestions for improving the technical 
components or quality of this system? 



b. Are there any other comments which you would like to make on 
technical features of this system? 
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Part B 

Introduction 

Communication via the CTS can be a two-way experience. It enables 
an exchange of information and opinions between individuals and groups 
in widely separated geographical locations. By completing this questionnaire 
you will help us identify some important educational aspects of satellite 
communications. 

Instructions : 

Complete all of the following questions except those which do not 
apply to today's satellite session. 

FOR EACH QUESTION WHICH" FOLLOWS, SELECT THE MOST APPROPRIATE ANSWER OR 
FILL IN THE INFORMATION REQUESTED. 

1. How many other locations were contacted by members of your group 
during this session? 

One (2:^; Two ( ) Three (2); Four (22); None (9). 

2. What proportion of those present at your location talked with persons 
at other locations during this session? 

All ( 1 ); Most ( 8 ); A few (39 ); Une person (6 ); No one ( 6 ). 

3. Did today's session actually start before the satellite communications 
began? . . K 

YES, more than 30 minutes earlier: ( 2) 

oYES, but less than 30 minutes earlier. (19) 
NO ^ (37) 

.4. * If your answer' to 3 was YES, which -of the foKowIno activities 
occupied most of the pre-sa tel 1 ite session? 

Explaining the objectives of the session. (9) 
Getting to know the other members of your group ( 1 ) 

Learning how to operate the equipment . ( 7 ) • 

Preparing content for the session ( 5) 

Reviewing material from a previous session ( 1 ) 

Other - specify 



5. Was there a moderator Qr group leader with your group? 
Yes 46_ No 13 

6. . Which of the sites 'talked' most during the session? 

Your si te / 

Another site sending picture (video) and audio 
Another site sending .audio only 
All sites 'talked' quite often 

Two sites ' tal ked ' actively , the others much less 

• O J 



(10) 
(21 ) 
(5) 
14 
7 
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7. Which of the following situations most resembles your impression 
of this session? 

Watching a TV program ( 4) 

Attending a lecture ( 8) 

Talking on the telephone (16) 

Taking part in a seminar (14) 

Listening to an 'open-line' radio show ( 4) 

Doing a 'live' TV broadcast from a studio ( 2) 

Having a conversation with friends (13) 

■ - Watching an experiment in communications ( 0) 

MARK ONE PLACE ON THE SCALE TO INDICATE YOUR DEGREE OF AGREEMENT OR ' 
DISAGREEMENT KITH EACH OF THE FOLLOWING STATEMENTS. 

8. It is always easy to tell who is talking on the system. 

Strongly ^ 18_ li5_ 1^ Strongly 

disagree - agree 

9. Participation by all members of the group is encouraged. 

Strongly J_ U_ 21_ ]9_ - Strongly 

disagree agree 

10. The telecommunications equipment used is a source of distraction 
during the sessions. 

Strongly 26_ 14_ _6_ Strongly 

di sagree agree 

n. The technical adjustments required. to establish satisfactory 
satellite communications take a long time. 

Strongly 25_ J4_ ^ _8_ Strongly 

disagree agree 

12. This session followed a closely predetermined plan. 

Strongly _6_ 1_8_ _9_ U_ V\_ Strongly 

disagree agree 

13. The system makes one feel self-conscious* 

Strongly 30__ j_0_ _9_ Vl_ J\_ Strongly 

disagree agree 

14. The telecommunications -system is easy to use. 

Strongly _J_ }0_ J4_ 3JI_ Strongly 

disagree agree 

15. There were embarassing silences during this session. 

Strongly 39_ i3_ J_ J_ Strongly 

disagree - • agree 
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In spite of geographical distance the different groups seemed close 
to each other during this session. 

Strongly _8_ i_2_ J_ 14_ 2i_ Strongly 

disagree agree 

It was more difficult to understand the material presented on the 
sate! 1 ite ^system than if it had been presented 'face-to-face'. 

Strongly 1 5 2] 8 9 8 Strongly 

disagree agree 

The cjurse of the session was altered as a result of participants 

questions and comments. 

Strorgly ]±_ 1_2_ li_ 1J_ Strongly 

disaqree ' - agree 

This session was too long- 
Strongly 2S_ 13_ _6_ _4_ .Strongly 
disagree . .agree 

One'does not get a good idea of how partici'pants-'a't other sites" 
arei> reacti ng . ^ ^ • ■ ' 

Strongly ]]_ r8_ ' 8 16 8 Strongly 

disagree agree 

Administrative and operational procedures occupied a disproportionately 
large^part of this satellite session. 

Strongly ^L. J- JL^ Strongly 

disagree . ' agree 

One gets a real feeling of personal -contact using t^is telecommunications 
system. - , ' ^ 

Strongly _j_ 2J__ 15_ Strongly 

disagree ^ • * agree 

The content of this session was presented in a confusing manner. 

Strongly pp 17 , Jl S 4 Strongly 

disagree " agree 

The two-way capability of the satellite system is essential for 
this type of teaching-learning session. 

Strongly ' _L ^ Al^ Strongly 

disagree agree 

The video (visual image) was not essential for this satellite 
session. 

Strongly 31 13 3 7 3 Strongly 

disagree ' agree 
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The 'content' of this session was very interesting. 

Strongly J_ 7^^ 9_ 23_ [9_ Strongly 

disagree agree 

It is difficult to talk to people in one's own group during the 
satellite sessions. 

Strongly Il_ Strongly 

di sagree agree 

There were times during this session when it seemed as if satellite 
contact had been lost. 

Strongly J_ _6_ J_ _7_ Strongly 

di sagree agree 

Most two-way communication which occurred on the system during this 
session appeared to be carefully planned. 

Strongly 2^ 12_ 1J_ J_ _6_ Strongly 

di sagree agree 

The person(s) presenting the major portion of the 'content* for 
this session seemed to be uncomfortable using the system. 

Strongly 32__ 18__ J_ J_ Strongly 

disagree " agree 

One gets the impression that mueh of the. sponta nei ty found in more 
conventional classes is inhibited using this satellite system. 

Strongly 1_5_ 15_ 1J_ 12_ J_ Strongly 

disagree agree 

I find it exciting to be part of an experiment in the use of new 
telecommunications technologies for educational purposes. 

Strongly J_ _4_ _4_ 1_2_ 36_ Strongly 
disagree agree 

Did discussion occur within your group during the satellite portion 

of the session which were not transmitted to other sites. 

Yes (24 ) Sometimes (26 ) Never (11 ) 



Rcie the -frequency with which each of the following types of 
activities occurred during periods of two-way comrDunication on 
the system. 

Asking for information 

Very frequent _4 1 7 21 1^ J Never 



Providing information 

Very frequent IJ 20 1_2 1^0 J Never 



Asking for additional explanation 

Very frequent ^ 14 17 1_6_ _5_ Never 



Providing additional ex pi ana t ion 

Very frequent 6_ IJ 14 17 6 Never 



A series -of unrelated comments 
Very frequent ^5 ^7 _9 21 13 Never 



Expressions of 


agreement 


wi th 


opi nions 


of others 


Very frequent 


5 


18 


14 


15 


1 


Never 


Discussion 














Very frequent 


9 


19 


11 


10 


4 


Never 


Arguments 














Very frequent 


5 


9 


4 


8 


27 


Never 



What is your overall opinion of the general level and quality of 
communication in this session? 

Highly 

acceptable 17 25 1 1 5 1 Unacceptable 



Do you have any comments or recommendations for improving this 
type of session? 
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COMMUNICATIONS TECHNOLOGY SATELLITE. 



EvaludtvDn Technical Log 
Instrument r2 

1. Date of Session: Day Month Year 

2. Time: From hours to hours 

3. Weather During Session: 

Clear Skies ( 3 ); Cloudy ( 3 ); 
Intermittent Precipitation (3); 
Steady Precip-? ta tion ( 3 ) 

4. If intermittent or steady precipitation occurred, please designate 
the type or types: 



RAIN: 



Light (.1 in. or less/hr.) ( 1 ) 

Moderate (.11 - .3 in./hr. ) ( 2 ) 

Heavy (.31 in. or more/hr.) ( ) 



DRIZZLE OR FREEZING RAIN: 



Light ( .01 or le^s/hr. ) ■ ( ) 

Moderate (.01 - .02-in./hr.) ( ) 

Heavy (.02^ in. ormbre/hr.) ( ) 



ICE PELLETS OR HAIL: 



Light (. visible on ground) ( ) 

Moderate (steady accumulation on ground) ( ) 

Heavy (rapid accumulation on ground) ( . ) 



SNOW: 



Light (visibility 5/8 mi or more) ~ ( 1 ) 

Moderate (visibility 1/2 - 3/8 mi) (2) 
Heavy (visibility less than 1/4 mi) ' ( ) 

5. Rating of video transmission during session. 
Adequate J 5 Inadequate 

6. Rating of video reception during session. 
Adequate 8 3 1 Inadequate 



Fating of aud"' o transmission during session. 

Adequate Inadequate 

Rating of audio reception during session. 

Adequate ]_ J_ Inadequate 

Rating of overall operation" of the systerT> during session. 

Adequate _5_ _5 Inadequate 

Were there any equipment malfunctions? (describe) 
Yes - 3 No - 9 



Were there any problems during interaction (question-answer 
discussion etc.) between students and faculty? 

Yes - 1 No - 11 
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COf':M'JNICATlOrJS TtCH:;OLCGY SATELLITE 

CRC Kan-Kochine Interaction 
Educational Experiences Evaluation 

Inslrurr.ent r3 

(For Participants) 

Instruction s : 

The objective of this questionnaire is to record your opinions of 
this satellite session. Use the following adjective scales to record 
yOL.r feelings about today's session. Complete each scale even though some 
may not s ?em appropriate. 

Please indicate your feelings about this session by placing an X 
on each scale. Check all scales in order and do not spend too much time 
on any one answer. 

For example: 

Active X Passive 

Please indicate how you would describe the relationship between your own site 
and the oiher si tes during this session. 



ERLC 



1. 


Equal 


■ 9 


7 


13 


14 


23 


Unequal 


2. 


Compet i t i ve 


19 


18 


14 


7 

•a 


6 


Co-operative 


3. 


Friendly 


12 


9 


10 


14 


19 


• Unfriencily 


Please indicate your feeltrrgs abou.t 


today 


's 


session. 


4. 


Long 


6 


13 


29 


13 


7 


Short 


5, 


Disorganized 


12 


22 


13 


16 


5 


Organized 


6, 


Relaxed 


8 


12 


17 


16 


15 


Tense 


7. 


Di ssc L i sfy ing 


17 


20 


13 


7 


n 


Sat isfying 


8. 


Warm «, 


6 


17 


19 


18 


6 


Cold 


9. 


Dragging 


3 


17 


17 


22 


7 


Lively 


10. 


Static 




21 


_25_ 


18 


4 


Dynamic 


11 . 


Good 


10 


16 


14 . 


20 


8 


Bad 


12. 


Useless 


13 


20 


9 


13 


13 


Useful 


13. 


Varied 


11 


8 


18 


. 22 


8 


Repetitive 


14. 


Product i ve 


_9_ 


16 


17 


16 


10 


Counterproduc 


15. 


Aimless 


J-O. 


20 


J 3. 


23 


2 


Directed 
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16. How much did you r.eet your own personal goals during this session 
Not at all VJ_ 26^ JJ__ _6_ Completely 

17. Was it clear to you what the objectives of this session were. 
Not at all _X .21L IL '^'^'O' ^^^^^ 

18. How much were the objectives of this session achieved in general? 
Not at all _2__' 20_ 25_ J15_ _5__ Completely 

19. How easy was conversation under these conditions? 
Very difficult _10_ _18_ ]^ Very easy 

Your name 

We are asking for your name in order to group tY.',^s questionnaire with 
others which you may answer during the satellite sessions. Although 
the results of your questionnaire will- be used, ycur name will remain 
confidential . 



Thank you for your co-operation. 
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COmUNICATIONS TEChTJOLOG Y SATELLITE 
Educational Experiences Eval ua tion 
' Instrument #4 

(To be con-ipleted by all persons 
involved in the planning, impl errientation , 
operation,, or evaluation of the 

•satellite project.) 

Name: , 

Institution : 

Position/Titl e: 

Function/Responsibilities in the Satellite Project. 



.Age: 20-Under (1 ); 21-25 ( i) 
j 36-40 (10); 41-45 ( 9) 



26-30 (10); 31-35 (19); 
45-50 ( 7); 51-55 ( 2); 
66-Over ( ). 



56-60 ( 2); 61-65 ( 3) 
Sex: Female (20) Kale (44) 

Educational Background (specify-^in terms of highest level completed) 

Elementary ( ); Secondary (8); College (12); University - 
Bachelors (1 4 );- Masters (16); Ph.D. (lO).M.D. (4) 

Major professional activities ' . . - 

Research ( 5); Teaching (13);- Administration (is); Technical* do) 
Student ( 3 ) . 
Other (specify) 



What is (are) your major area(^) of specialization? 



V.'hat are your reasons for participating in the CTS project? 
(Check all those in the list which are applicable.) - 

Required to participate by supervisor . * ( g) 

C^.-wwrjged to participate by supervisor ( g) 

Encouraged by colleagues ( g) 

Personal and/or professional interest (27) 

Professional devel opr;ient ' ( 5) 

Possibility of personal 6dvanco;T^ent ( ]) 

Substituted for a colleague (7) 

Other (specify)'- ( ) 



When did you start working actively on tre project?" 
Month Year 

Had yo'j ever used audio-visual equipr::ent before becoming involved 
in the CTS project? 

Yes, often . 2. never' 

Had you ever worked with telecommunications equipment before beconin 
involved in the CTS project? 

Yes, often -^^p -ji No, never 

In your opinion which of the following factors is the most 
important criterion for measuring the success of this project? 

Technical feasibility {J) 

Satisfaction of the student ( '•^) 

Satisfaction of the experimenters ( 1) 
The relevance of the project to the institution's 

objectives and/or rnandate (22) 

The ptrssibTlity of continuing the project , (2) 

An innovation in teaching/learning" (16) 

ether (specify) ( 2) 

Do you ihink you received enough training for your participation 
in the CTS project? 

Yes 42 No 17 



Which of the fol 1 owing persons or groups provided assistance in the 
planning and implementation of this CTS project? 

Your institution's audio visual department 

Yes 36_ No _9_ . 

Your departmental colleagues. 

Yes _41_ No 

Your departmental chairman, dean, vice-president. 
Yes ' 1_4_ No 1 7 

Your colleagues in other departments. 

Yes 23^ No 

The CTS evaluation team. 

Yes 26_- No 28. 

A university; corrirm'ttee 

Yes- 7 "No""' 24 



16. Cont'd. 

Other{s) (specify) 



17. How many of your departmental collea^gues are associated with this 
project? 

'Don't know ( 4); 1-5 (34); 6-10 (2 ); 11-15 (1 Or 

, 16-20 (12); More than 20 ( 3). . * • 

•rv ' ^ ' ' • 

18. What proportion .of your working time is dedicated to the CIS. . ^ 
project? ' 

0-25% (31); 26-50% ( 8); "51-75% ( 8); -76-100% (7 ). 

19. If your participation in tha CTS ^project is not , part of your regular 
duties, have you been relieved of your other ,^duties for the 
duration pf this project? 

Yes 2 Sometimes 6 Never 49 • ' _ 



20. If your answer gto 19 is never, how much do you estimate that your 
involvement in CTS has increased your work load? 

^100% ( ); /5%; ( 3); 50% (7 ); 25% (23). - 

21. " In- your "opinion is this projectomostly the result of a team effort 

or individual efforts? 

Mostly * o 

a teatn 37 10 3 5 5 Mostly individuals . 

22. Oo you think this project meets or addresses a real need? 
Yes _56- No _3^ . . : 

23. Do you think this project will be able to continue after the end of 
? the satellite experiments by using other resources? 

V 

^ Ye^ _27 No _15_ ■ . ' 

24. Rate your degree of satisfaction with your part in this CTS project? 
/ . Very satisfied JJ^ 22_ j6_ _2_ _ Quite dissatisfied 

Comments on your satisfaction/dissatisfaction 



Do ycu have any comments which you would like to make about your 
involvement to date in this CTS project? 



Please make any recommendations that you think could improve this 
project. 



COMMUNICATIONS TECHNOLOGY SATELLITE 



Educational I'xperiences Evaluation 

Instrument irS 
(To be completed by participants) 

Introduction 

The project which you are participating in is one of a series of experiments 
examining uses of the Communications Technology Satellite for Education. It would 
greatly help us identify some important educational aspects of satellite communications 
if you would complete this questionnaire. 

Instructions : 

Complete each of the following questions by selecting the most appropriate 
answer or filling in the information requested. Please indicate your feelings about 
this session by placing an X on each scale. Check all scales in order and do not 
spend too much time on any one answer. 

1 . Name 

2. Age: Under 20 (6); 21-25 (3); 26-30 0^); 31-35 (9); 36-40 ( 7 ); 

41-45 (6); 46-50 ( 4 ); 51-55 ( ); 56-60 ( 1 ); 61-65 ( ); 
66-Over ( ). 

3. Sex: Female (26) Male (24) 

4. Educational background (specify in terms of highest level completed). 

Elementary (3); Secondary (7); College (14); University - Bachelors (14) 
Masters ( ICT,; PhD. ( 1 ). M.D. (1 ) ' 

5. Where is your home? 

In a city or town (45); In the country, but n' t on a farm (3); 
In a village ( 1 ); In the country on a farm ( 1 )• 

6. Date of this satellite session. 

Day Month 

7. Were there enough TV monitors in the poom for the size of group watching this 
session? 

Yes 35 No 15 

8. Were there enough audio systems for the size of the group watching this session? 
Yes 46 No 4 

9. Were you. always able to clearly hear everything that was said on the system? 
Yes 34 . Sometimes _1_0_ No 6 

10. ^^r^vi t;'Ould you rate the physical layout of the room used for this session? 
Excellent 7_ 16 18 6 1_ Poo^ 

11. V.'ere the TV Monitors satisfactorily arranged so that the video portion of the 
program could be seen by everyone? 

Excellent 6 21 11*4 8 Poor 
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12. How good was the general quality of the TV picture for this session? 

Excellent 32_ 11 5 2 Poor • 

Picture quality - picture quality 

13. Rate the overall adequa.cy of the TV picture for achieving the teaching-learning . 
objeotives of this session. 

Adequate 17 18 10 2 Inadequate 

14. How v;ould you rate the volume of the sound? 

Too Loud ^ 7 32 9 _1 Too Low 

15. Rate the overall adequacy of the sound system for achieving the teaching-learning 
objectives of this session? 

Adequate 20 9 15 5 1 Inadequate 

16. Taking into account the teaching-learning objectives of this session, rate the 
overall adequacy of this communications system for achieving them. 

Adequate 21 14 8 4. Inadequate 

17. VJould you like to regularly use this type of communications system for taking 
similar courses? 

Yes 30_ No ^6 ' 

18. Which of the following situations most resembles your impression of this 
session? 

Watching a TV Program ( 4 ) 

Attending a Lecture ( 1 ) 

Talking on the telephone ^ ( ) 

• Taking part in a seminar (11 ) 

Listening to an 'open-line' radio show (5) 

Doing a 'live' TV broadcast from a studio .09) 

Having a conversation with friends - ( 1 ) 

Watching an experiment in communications (7) 

19. It is always easy to tell who is talking on the system. 

Strongly 2 9 8 19 12 Strongly 
Disagree Agree 

20. This session followed a closely predetermined plan. 

Strongly 8 14 • 14 12 Strongly 

Disagree Agree 

21. In spite of geographical distance the different groups seemed close to each 
other during this session. 

Strongly 10 11 15 14 Strongly 

Disagree Agree 

22. This session was too long. , 

Strongly 14 13 10 8 2 Strongly 
Disagree - Agree 

23. The 'content' of this session was very interesting. " 
Strongly 1 4 12 17 16 Strongly 

Disagree Agree . . 

24. Kost two-way communication which occurred on the system during this session 
appeared to be carefully planned. 

Q Strongly IQ 13 8 12 - 6 Strongly' 

ERIC Disagree ; Agree-, 
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25. What is your overall opinion of the general level and quality of communication 
in this session? 

Highly 2L_ ^-^ 6 _2_ Unacceptable 

Acceptable 

26. -Do you have any comments or recommendati-ons for improving this type of program? 



THANK YOU FOR YOUR COOPERATION. 



ERIC ^ ^ 
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APPENDIX B 



COMMENTS 



ERIC 



COMMENTS 



INSTRUMENT #1 
PART A 

More participation by moderator to limit speakers dialogue rather than letting 
them ramble on and on. 



Consider the creation of two smal 1 er groups of 7 or 8. Each group would have 
its' own monitor a_nd two microphones. This structure should facilitate the 
invol vement.of more reticent individuals. Pre-establish time limits for your 
speaker's formal presentation and encourage them to avoid rambling responses. 

When phone calls came in one lost continuity of program through phone inter- 
ference. Jime allotted does not allow sufficient questions and answers. 

More microphones. Technical advisor always on hand in each communi'ty to cope 
with technical difficulties. 

To let our cable system viewers at home see the operation in the studio via 
television on the cable system, so they would get a more complet.e idea of how to 
operate audio talkback to other stations. Give a little more time to make 
adjustments to audio and visual aspects in system from receiving first visual 
transmission until on air time. 

Panel too large. Two panel members would suffice. Questions period thankfully 
generous. Video tapes of just the right duration. Good progress. 

A better cueing system for people who want to talk on the system. 

The man came agross equally well on the taped and studio portion. Didn't 
know how to finish a close. The v/oman came across much better during the 
'studio interface.. During the taped portion the monotone voice and lack of 
facial and/or hand i.mpression proved irritating to the point of my mentally 
tuning out much of what was said. 

Although it may be unfair to compare with commercial television, the cajnera 
work and the voice qualities of the presentors occasionally interferred with 
the message. 

If possible, dry runs before show to ensure all locations receiving and trans- 
mitting. 

We would hear the noise from the slide projection used in the VTR portion of the 
program. Needs a better "phone in system" for cable viewers. 

Please flash speakers name on more frequently, some speakers are never named 
other than the initial introduction, as a member of the audience I like to 
address someone by name. Very good - no suggestions at this time. 
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Local animators should have a handset socket or similar equipment to connect 
both circuits of an incoming telephone call directly into +he program feed 
during interactive period without having 'to use^microphone and button. There . 
are times when the video is not needed and the system could be broken down into 
800-1200 audio circuits to allow more individual interaction, or to allow 
"yes-no" feedback into a computer for percentile readout to the lecturer. 

I would like to see video reception from other participating localities 
perhaps at times using split screen composition. 

The session was too long in the initial stages. 

Easier access to microphones. 

Other centers questions could not always *be heard adequately. Sound quality 
could be improved. 

An addition of microphones would encourage more active participation. I find 
it terribly unique which will prove to be a great improvement to educational 
facilities. The sound volume was at times too low though. 

« 

One tends to forget that the technical aspects exist. 
Keep salesmen for equipment off the show. 
PART B 

Studio audience in number would not be necessary another time. It is difficult 
to find people to participate. Telephone is good. Less programme and more 
audience participation. 

Program should be longer. 

Some of sort of training for studio staff and participants so that interruptions 
such as telephone calls, mike problems, etc, are handled more smoothly. 

Too much time allotted to video and primary station and too little time for 
questions by phone or studio audience. 

By the time the phone calls reach the studio and put on air, quite often the 
topic has changed. A faster way of handling the calls would make an improvement 
in the interaction of the outside viewers. 

I feel if the purpose of this program is interaction, more time should be given 
to the viewing audience. I found the speakers to be very good and I agreed with 
most of what they said but I feel they expanded too much and got carried away. 

The two speakers seemed to elaborate on answers too much wasting air time and 
cutting off further questions. Here in Kelowna there were at least twice as 
many questions as were actually asked on the air. If possible would be nice 
to see the person from other centers asking questions. Would bring the centers 
closer together. 

Lack of human feeling in the program. Base the topic of interest on ideal 
grounds with central personalities free of prejudice and rigid preconceptions. 
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Talkative panel at the studio di?:ouraged group participation at other ^i tes . 
A discussion session should begin half hour before actual broadcast, in order 
for audience to better develop its' ideas and questions. A floor director and 
the panel moderator must attempt to prevent guest panelists from squandering 
valuable air time with excessive discussion of questions. 

More time for site to site discussion. 



I feel that there is too much influence by the Vancouver panel members. It is 
diffici>lt for external panel members to get involved. Probably fewer Vancouver 
members would help with support by more external rnembers Under the present 
system it becomes too "Coastisized" , more interior involvement is needed and 
additional time for preparation would be desireable. 

' Reduce the size of the panel 1 

This has been one of the better programs from BCIT on this subject. (Forestry - 
Harvesting). The panel answered what they were asked and did not ramble on. 
A fair amount of interaction provide for interesting responses. 

One thing that was annoying was when two people tried to speak at the same time. 
Perhaps this was a breakdown in communications or a lack of organization. 

It would be nice to have the people at the different sites who are making 
comments to be on the air also (visually) so that you can see who is speaking. 

A little more time might loosen up the participants and allow for better 
discussion. 

More distinct answers from panelists. More interaction time. Would have 
helped to have and stateo instructionc'il objective. 

Did not feel inhibited by this type of session, particularly liked the "call 
' back" number given by SFU so you could ask questions after the time was used 
up. Panels seemed very human and very approachable. 

r 

It was too short a time to share opinions, since there were four stations. 
So it is better to have two or three stations for the program. But this is 
dependent on the programming. 

Difficulty cutting in to ask questions. 

It would be more interesting if the group met earlier than the broadcast, 
introduced themselves, and perhaps even had a short discussion on the subject 
to be aired. 

Must ensure that panel on the other end can cover all bases. I found that 
"Forestry Careers" panel was too slanted toward coast forestry. A cormient to 
take effect produced indignation within the panel. 

Provide a broader range of "experts" on the panle. 



More interaction - better pacing. 
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I 
f 

Spend more time and thoughtfu i ness on consideration of motives and principles 
of justice and less on attempting to reach a pre-determirfed conclusion through 
massive accumulation of facts and supportive argument. 

In many cases opinions or arguments could not be given as information was 
being given. 



I find this an extremely acceptable and highly approved method of corr.muni cation 
for teaching purposes in education media. Hopefully it shall continue. 



INSTRUMENT 

More t^me is needed for the questioning time. 

The Satellite Education programs are a strong contribution in my personal 
pursuit of knowledge and even though I have generally responded to questionnaires 
on the program I have enjoyed, it is not to detract from the overall benefit 
derived from the programs. It has been the result only of chance. 

Panel was too biased. Panel dominated too much* 



INSTRUMENT jS^4 

Greater emphasis has been placed on the technical accomplishments than oh 
educational values. Right now CTS is an expensive toy - a tool whose place 
in the educational system has yet to be established. 

Now that the experimental period is (far me, at least) over, it is imperative 
that someone or some group attempt to get guidelines for the kinds of needs 
that can best be satisfied by CTS. So far CTS has been a solution looking for 
a problem, v;hich might be okay for an experiment but in my opinion sets a 
dangerous pattern which should not be allowed to continue. Please can we have 
some medium and long range planning. 

Once the project was actually delivering programs all went relatively smoothly. 
This indicated I think tnat the event and planning in which I had a rple was 
well, if hastily done. This was quite satisfying. 

Thank God it's over! 

In the long term if satellites are to be used to deliver interactive television ~ 
then it is essential that only programs suited to this medium are delivered. 
These would appear to be specialized, other tehcnical, programs aimed a specific 
audiences, e.g. the physician upgrading and the librarians data bank access 
programs. Interaction must take place throughout not just at the end of a 
standard one-way program. Conference calls after a simultaneous cable cast 
can do that. 

More lead time would have greatly improved the quality both technically and 
academically of the program series on forestry. Six \-?eks is not enough time to 
produ^'? one program, let alone an eight part series. 

As a public librarian talking about old books, their heritage value etc. I 
received letters from people watching the programir.e on cablevision with title 
lists to check. 



Thoroughly enjoyed myself. Good potential for public libraries as a com.'nuni cation 
' device. • ' 

I do hope public librarians again have the opportunity to 'participate in\ future 
programmes. Public libraries have an educational role to' play and in thtir 
programming can reach the person in the home better than in the ■ educati onal 
institutions. 

I would ha^'e liked to see a short (2-3 min-. ) video tape of how looked and 
sounded on screen. I had no idea whether the rate of speed, expressions etc. 
were effective. 

Although quite satisfied with technical performance program content of some 
. consortium members was either very poor or aimed at wrong market (persons). 
^ Needs study on this aspect required. 

1. Needs study to detemine content and identify markets of different types. 

2. Encourage consortium members to improve their programming. 

3.. Animators should be trained in systematic techniques to solicit r'esponses. 
from other sites so that the sites do not all talk at the same time. 

4. Krovide audiences with information about how much time is left close to end 
of X program. 

5. Utilize information from STEP evaluation study and a Needs Study to plan 
future programs/area. 

6. Explore different areas (ministries) re continued fundi ng/requi rerr.ents/ 
services etc. 

We need a longer lead time and longer experiment time in order to do real 
identification. 

.Better animation and much clearer identification of needs (matching audience 
with subject). 

My main dissatisfaction stems from- the fact that few viewers understood it was 
an experiment. . 

Many last minute changes in organization - but that kept it interestinn. 

The project could have been shorter. Most of the questions re technical 
feasibility and relevance to future mandate (possible) for PEMC were probably 
clear after the first half of the project. However 

The project should have been of greater duration. A longer time period would 
have allowed complete courses to have been conducted. We should have utilized 
the Saturday time available to use for broadcast of PEMC> producti ons , arts 
and culture, and other interesting documentary materials. 

Interesting and informative. 

Better equipment and coordination of audiences to reach more people. 

Satisfied with production on PEMC's part. Feel there is a need for better 
planning and packaging of the various institutions insert material. 
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Felt that vn'th better lead time I could have contributed to over-all quality of 
insert material. 

Time ^^)d money.. 

Satisfied to ^see how well PEMC was able to cope with such a hurried complex 
project. 

Sorry I wasn't able to contribute more. 

More lead time. Hore understanding by outside people of the technical and 
administrative difficulties involved -in organizing such a project. 

I ./as satisfied with what I was asked to do, but whether the use of graphics 
could have been more creative or meaningful in conveying information or viewer 
satisfaction I do not know since the subject was never broached. 

My involvment was only in supplying needed titles or graphics upon demand. 

As a creative element, rather than, a service element there is conceivably much 

more that a graphic designer could contribute to a project of this nature. 

Having not been that closely involved in the production, I am not able to make 
- value judgements. 

PEMC should have more control over the technical specifications of programs 
aired. 

An overall look at the job I had to perform for this project had a very slow 
response. Due to the nature of the questionnaires the participants co-operation 
was limited as the questionnaires tend to be too long Towar(;b the end of the 
project more co-operation was noted and this helped the evaluation become more 
meaningful. 

1. Kiore co-operation with the animators and the studio. 

2. Longer time to interact. More opinion and less questions from the 
audience. 

3. BCIT site should have been set up for interaction as well. 

4. Better cueing system for interaction as too much time was lost waiting 
for the site to respond to questions. 

5. Better understanding of the program -content. Preparation before the 
program so that the key audience would be in tune with what the program 
content was and could prepare their questions before hand if they were 
not already answered in the discussion. 

0 6. Smaller or more variety of expertise on the panels with only the top 
dogs on so that the questions could be readily answered adequately. 

7. Do a study before the next project is launched to check the needs of 
the community. 

8. Check education areas of NEED for Distance Education. 

Our program could have been very useful if studio rehearsal time had been 
allocated. 

i^iui* the help of the technical people at SFU and PEMC the program v/ent extremely 
well. I feel that the material presented stimulated some thoughts on the subject 
area and successfully dealt with the associated problems. 

My involvement was'short term. Once the program was completed I did nothing more. 
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The most important aspect of the program is to satisfy a need -in the comjTiun'i ty . 
If we simply televise programs without looking at each community it is a waste 
or time. The animators should have tinip to dS^<;i^5, Ui^j h'iOds of the community 
with respect to the specific topic suggested by Lhe institution. 

The technology was more useful than I had expected. 

Much more lead time and funding for - local animation, production, pre-work, 
materials. More careful selection of content for this format. 

It was of great interest for us to take part. 

Questions and answers should be kept precise and of interest to tall. 

Satisfied in bringing education from the University to the community. Very 
worthwhile. 

You should attempt to sort out the video delay problems and allow people to 
produce a full video tape in advance. Visual material has a definite impact 
but -it must be organized well, I suggg^t that keeping people who are speaking 
off the screen and providing directly applicable visual aids, is the best 
format. It's amazing how the*general populace can learn to appreciate graphs 
etc. even though their first exposure may bemuse them. Now that you have captured 
the audience, demand something of their intelligence'. 

Panel Moderator and floor manager openly disagree during program re: introduction 
and time left at end of the program. 

Satellite pins seetned a w^ste of money. 

f need more experience j>^fore 'I am able to teach effectively by this method, 
but I can see terrific potential for this type of course delivery in outlying 
areas, of the province. 

IV has been an exciting and valuable experience. 

Video projection seems an excellent way of delivering courses to smal 1 Communities 
in B.C. Should be expanded. An overall plan, includi ng. needs , interest assess- 
ment in the communities to be served, the rationale for us i ng . satel 1 i te video 
rather than local college resources is badly nee<ied. Professors need time and 
some training to enable them to teach effectively on T.V. 

I felt the filming in Vancouver Was poor and the taping at UVic was better. ^ 
The 'animators played too dominant a role; the "^rterested people in the receiving 
community weren't involved. If they weren't interested why have the program. 
If they were then animafors should have involved them. 

My involvment was minimal but I feel our program was too'skilled and very much 
geared to para professional in Family Law, social workers, court-personnel, etc. 
who: didn't take part. 

>/* Fewer people involved in end product. I*m a great believer in a small 'and 

efficient team. I feel, especially in Vancouver, that resouj;X5iS--W€f^e 

being wasted in excess studio staff. — — '""'^ 
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2. The animators need to be more involved in community needs - setting 
programs that are requested and to which local people rr-ake commitment. 
Should not try to appeal to a general audience. That's better done using 
cable T.V. 

3. Media professionals like the excellent fellow from UVic should direct more. 

Technical work was good, studio cooperation was good. Preparation good. 

. Less type and more concern with content. Too much concern with media personalities 
i.e. the absurdity of the LaPierre-Watson exchange. Too little with intellectual 
content. 

Why butcher the language by calling this document an instrument? Our project 
should have been tighter. Example: more clearly identified audience, good 
enunciation etc. I have impression entire project was created to fill 
satellite time rather than to fill some other need. 

I found the work interesting, but was disconcerted by having the only rehearsal 
scheduled at the last minute for a time I could not attend. Lack of knowledge 
about the interest and composition of the audience made planning hard. 

Continuity of programming, a planned for audience, time to develop meaningful 
themes and an adequate dialogue. Avoidance of th^ television format of instant 
communication of small, isolated facts (or fragments). 

The communications concept was most satisfying in that it was such an improvement . 
over "open line" shows. We received good, spontaneous questions that were 
relevant to the! discussion. 

My involvement was a relatively few hours for meetings, taping sessions, and 
the live portion. Td be willing to do it again. 

More attention could be paid to colour of background panels. Taping at UVic 
showed brownish panels and all panelists also wore brown tones. 

Time period to short (1 hour). Some confusion in roles of chairperson of panel 
(myself) and the roles of introducers (Carney, Fotheri ngham) , too many cooks. 

A learning experience, intrinsic reinforcement in seeing thefinished product, 
team (prograrmiing and technical), I assembled turned out to be competent, 
.congenial, and interested in the project and its' implications for education, 
generally positive and supportive relationships (working) with DEPG and PEMC. 

Limited number of colleagues interested in the project - majority of those not 
interested either uninformed or felt all that project did was re-invent the 
wheel. 

Longer planning time, better summaries of work of others with such projects, 
personal on site evaluations (simply didn'_t have time to travel to one or more 
sites), better liaison with^J^^Tmmators (probably a function of lack of 
time) small group^worJ^-hCps on effective use of systems, including program 
oes fgrr an(JliTa7ke t i ng . 



Preparation of audier.ces in locations poor/our particular project could be 
carried out just as well by telephone. Elaborate technology not nec-ssary. 
There is, hov/ever, a real need for this type of service in the community 
C0.1 leges. a 

Interesting experience, but f ra\g+rr^.with problems and frustrations. 

More long range planning necessary. Communications between overall organizers 
. of project and distant locations could have been much better. He were asked 
to participate at a very late date which made it very >difficult to get a program 
together. It also seemed th^t our program was misrepresented to the audiences, 
they were expecting something quite different from what we presented. 
Presentation of lectures via satellite is feasible, but reference service cannot 
be given via satel li te. .^he satellite ca/inot replace library resources in 
distant locations. The animators did not/ prepare audience v;ell due to .poor 
communication. . Most training we had was due only to our own efforts; not - 
due to training provided for us by people at the Hermes location. Susan 
Leslie was very helpful in coming to U.B.C. to help us with our project. 

Preparation in the outlyi ng areas was very poor. Our particular project was a t 
waste, of resources. The one positive aspect was personaV interest in the 
project and the learning experiences of how reference cannot be done via satelli'^te. 

The librarians from the colleges who came to Vancouver and met with us understood 
our program. Those who had to depend on the animators were unprepared and 
misinformed about our prograni. 

1. The satell/ite appears to have possibilites for lecturing and teaching but 
not for answering individual reference questions. _ There is^no substitute 
for basic core collections at the colleges and for strong reference 
collections to provide students with the tools tO'-access materials available 
for inter-library, loan from the lower mainland libraries. 

2. Better preparation -and- communication with the outlying regions is needed. 

3. More time for rehearsal. in the studio would have been useful. Fortunately 
we had prepared and rehearsed our program before our arrival at the studio, 
although we had been toTd v;e would have time to renearse at the studio. 

■I was very glad to have immediate "feed back" from the comments we were making. 
This uvo-way of communication is ideal - you know what you have successfully 
taught and what you need to reinforce'. 

I was surprised at the level of ability of many of the people who responded. 
They wer]e very bright people. 

I thought that we could have had a better presentation if we had had a little 
graphics work' done. , We needed a few charts, the odd picture, etc. to jazz up 
our presentation.. Only very patient people could have followed the fuzzy out 
of focus charts we used r graphics could have made things much easier to under- 
stand. 

High degree of cooperation with Health and PEMC groups. New adventures |n 
production. Reach out to the rest of province. The set at PEMC was bee .tiful 
but* inflexible and possibly somewhat intimidating to non urbanites. 

Kiore and better lead time required. A greater means of contact with local 
animators. Less rigid time slots for certain programs. 
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1. Let audience know what degree of participation (interaction they can 
anticipate - reduce expectations) where necessary. 

2. Provide a room adjacent to receiving area for immediate pre and post 
TV planning and follow up at local level. 

3. Have more local animators so the job distributed. 

4. Insist cn no more thar. 10 minutes preferably 2-5 minutes, for any production 
segment before a break. Send longer stuff ahead of program via videotape. 

5. Get audience size and I.D. information before program starts. 
G. Let content determine length of program. 

7. Provide a more versatile set. 

8. Save satellite transmission of TV for those things it can do ..hich other 
things can't do. Some possible critei^'a; of immediate benefit to more 
than one local group: where interchange among groups helpful; where 
citizens can get vital information regardless of their number, because 
of inavailability of teacher, doctor, consultants. Obtain research data 
on this interactive element from proven performers e.g. country check up; 
hot liners etc. 

9. Encourage their local production to the city on topics they know most 
about. (Audio first, then as technology permits video.) 

10. Set educational objectives and evaluative instruments up at time of planning 
programs. This means 3-4 months lead time. 

Finally thank you Pat, Gene, Arvid, Bernie, V/ayne, Cathy, In spite of 
suggestions above - a herculean task was performed under the most trying 
of circumstances. 

Not enough lead time to detennine the needs of the learner. Method should be 
developed to illow.more interaction, if necessary, I doubt that interaction is 
such a necessary part of education. 

Studio set should be flexible. Most program content was diluted as it was 
aimed at general public. Programs should specific to leariier groups. From this 
experiment it was obvious that in many cases the use of live TV was not a 
necessary part of education, many subjects could be handled by video cassette 
sent to the local areas followed by a telephone conference. It is my Reeling 
that a in-depth study of the use of television in long distance learning has to 
be made and new rriethods ot the use of this media established. 

The build up to the presentation was quite professional but the actual present- 
ation (the most important) ran into serious problems on the air - due to a 
conflict between stage managers. 

Better teamwork and coniTiunicaticn at the management studio level. 

Much more production money necessary to up-grade the level of skills in all 
faceib, especially in the content of the programs. 

I could be of greater use to the project by being involved at the program 
concept stage as well as the production phase. 

More independent media production people to produce and package the material 
on behalf of various institutions. Clearer objectives for the project which 
will enhance the weight of the individual programs. A highly visible execproducer 
type to co-ordinate the aesthetic and educational content of all programs. 
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A bigger comni tment to the rural viewer by delivering uiore than a one shot, 
e.g. some general interest immersity course on perhaps B.C. History that via 
a correspondence program will earn university credit or perhaps the delivery 
to the 'system* of special interest r.iaterial like a course on 'occupational 
health' delivered to industrial work sites, logging groups, and remote 
settlements. Also, program from the boonies to the city and seat of government. 

Its results can now be applied to post secondary education for all in B.C. 
who are isolated from colleges and universities. It offers the opportunity 
of broadcasting video tapes of the internationally know scholars brought to 
UBC by the Cecil Green visiting professors and others. 

This was definitely a learning experience for me, but I was gratified to find 
the interaction with the outlying corrmuni ties was the 'highlight' of the program; 
and their response to our panel discussion was most satisfying. 

An enjoyable experience! Thank you. 

Inadequate communication to local areas re its availability. Terrible colour 
recording. Unable to see director (should wear white). 

Personally would like to have done a better job as panel moderator. Received 
great support and help from studio team at PEMC. Error in control room threw 
"me a little in earlj stages of November 22 program. 

Opened up not only world of satellite conmunications but a subsequent telephone 
conference with Dawson Creek audience because of transmitter problems the night 
v^e produced our program. 

Interesting experiment. Delay interferes somewhat with 'free back "and forth 
discussion. Time limitations would bS interesting to have visual both ways. 
We had on one occasion a nurse in one of 'the communities asking some technical 
questions and I am sure the discussion was relevant and helpful as well as the 
lay participation for which it was devised. 

The use of the key resource person in the community also very helpful and able to 
coordinate questions from listeners effectively. 

I think this projects-points the v;ay to a very effective means of disseminating 
specialized instruction and information (which tends to be concentrated in the 
large urban areas) but the ways it failed or fell short are related mostly to 
the project being too dependent on the "Broadcast Television" model. 

I sound all the PEMC staff very friendly, co-operative and competent (with 

one incident excepted - a VTR insert that should have been cued up and wasn't). 

Most of our troubles were with factors of time and place. As in broadcast 
television, our on-air time was divided into discrete one hour segments; no 
attempt was made to discover an appropriate duration for an interactive 
health-problem discussion. Considering the amount of physical effort that 
participants (on both ends of the system) had to put out to attend, they were 
very likely to feel short-changed and frustrated; the technology made the 
experts appear accessible, but the 53 minute thinking effectively denied this. 
The not enough time feeling (which increased with complaints from frustrated 
participants) coloured our approach to everything. This was often counter- 
productive as w^en it was felt that we shouldn't take time to warn up or establish 
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rapport between and among sites and participants. 

As in broadcast television, the design of the set see-Tied tO" say "We're a bunch ^. 
of slick TV professionals, this is a big deal, and we're going to zap it to you." 

The flying saucer motifwas very cute and quite intimidating, certainly to 
some panelists, and probably to m^ny tiewers. It is quite daunting enough for 
scTieone to appear on television for the first tine, without deliberately corr^poundi 
their discor.fort and anxiety by rounding them - floating them in a 'dark sea 
on an unnecessarily high riser. - also question whether it was wise to confine 
cameras from head to toe. 

In other words, I would recommend that ways be found to loosen up the mood or 
mind set, to find and eliminate spatial and ten^poral constraints; access, 
participation, dialogue across distances could all be eased. Let the medium 
serve the people rather than the people serving the medium. 

Satisfied on a technical basis. 

Programs were generally lacking in prepration. Preparation is the essence 

of television programing. Make S'jrn the guests, the crew, the hosts understand 

why and what they are supposed to dc. 

Kake the best use of live demonstaration and interaction, as well as guest 
V.l.P.'s. I do nut see the point of using valuable satellite time for playing 
tapes unless thoy are brief and could not be done live. 

How about an interaction program where the' V.I. P. s are at regional studios 
(unseen) and the audience. Guest7^-ns him or them from the Vancouver base. 

Don't compete for the general public by programming in TV prime time. 

The people putting the programs together must be flexible. They must be 
willing to change the program and i t: format to fit their audience. They must 
let the other centres speak and answer the questions directly, not beat around 
the bush. Any panel must be kept to a small size and the idle chit chat cut 
out. Television is a visual medium and to keep the centres interested during 
a lecture or speech, slides and other important data must be shown, not just 
talked about. The studio audiences may have been large had the sessions been 
able to be in the college, the real objective of the CTS project. 

Owing to ignorance an'd lack of experience I v.asted appreciable time in following 
what transpired to be dead ends. 

Increased publicity. to encourage public participation. Tapes made available 
for showing in other areas. A more practical procedure for receiving and 
stacking up incoming calls and questions. Better equipment for receiving and 
transmitting telephone calls at the regional studio. 

Sufficient funds available to produce programs. Better publicity - increased 
public awareness. Improvement in technical ability to receive incoming calls 
stacking capability. More direct coniiiuni ca ti on with satellite groups re 
program content and format. 
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Very satisfied from the point of view that my contribution helped to make the 
project a success. Also I am very satisfied that I have gained new knowledge. 
Very unsatisfied from the fact that my efforts were not in anyway acknowledged 
by my superiors. 

It would be nice to have the college recognize the investment of time and self 
motivation that I personally committed to the project. 

Better commurvication at the college senior administration level and the Kinistry 
of Education. Better communication at the project level - too often there were 
too many cooks and the left hand - right hand syndrome emerged. More 
preparation time - technically and program wise. Time on the satellite system 
just to experiment or even play with various modes or interactive TV. I felt ^ 
that we tended to track into only one format of interactive TV out of perhaps, 
many possible formats. 

INSTRUMENT #5 



The set is over powering, too much time spent contacting outside groups. 
Add two-way video. 

flore interaction between panel members. 

Panelist should speak in laymans terms as the majority of the audiences are 
local peoples. More interaction time. 

Stating objectives of the program at the beginning.^ 

Brief description of the medium being' used or has this been done? What 
satellite, where is it, is it Canadian, how old, how does it work? What 
is the heart foundation? Who are the people, are they family oriented, single? 
Some personal background. Make them more acceptable to the audience. therwise 
an effective program of real interest. 

Suit the programs for local situations. 

Very Triipressed with PEMC. Important to have good videotaping, otherwise could 
be dreadful. Two-way communication makes session much more interesting- 
Lectures are boring on screen. Hard to focus on printed material on a screen. 
Perhaps these materials should be avoided.' 

More use of the two-way communication facilities - allow receiving stations more 
time or a video signal showing the receiving stations. Try to show less of 
the typical television talk show format and a little more the informal two-way 
audio and video communication. 

More! ! 

It would help to have some kind of handout to keep all aspects of the program 
straight. 

Closing discussion of probleins related to techriological development most 
interesting section. 
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There is too niuch lag between a person starting to speak and the camera 
including them in the picture. Many questions in this evaluation form address 
themselves to the teaching/learning objectives, but at no time during the 
program were they stated. How is it^possible to evaluate the unknown or was 
it supposed to have been so obvious? 

Too cnany people talking. I found the audience too noisy, difficult to hear 
the progrciTi because the floor director was talking to the panel while the video 
tape portion was on. 

Have interactive v'deo as well as sound. That would rise the level of interest 
Perhaps at this experimental stage the content is not too critical but this' 
was a fairly dull cablevision type TV program at very limited interest. Some 
really imaginative programming, utilizing the remote locations much more would 
be a big improvement. 

More conversational communication from participants - who they are, wher^ they 

live etc. With only visuals from the studio a greater sense of the oth 

people should be achieved. Communication is more than just asking que jns. , 

The use of the satellite technology in education can only be as effic; t and' 

effective as the quality of the education process. In this particular 

program on arthritis I found it hard to imagine the video insert material being 

of much interest even to arthritis sufferers - the questions did not often 

seem to be connected with the video material. Some soliciting of opinions 

from the outlying audience v^ould have been interesting. The people issues . 

re diet, acupuncture, verbalish were not adequately answered by the ' es tabl ishrient' 

medical panel. Why the time limitations which seemed arbitrary? 

As a tax paying citizen who will not profit by this type of corrir.unica ti on , I 
would say that the system for said communication is very good. If I was to 
live in the community where there programs are transmitted I would surely 
watch. 
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RE CO:-:XEN DAT IO N'S 

PRODUCTION 

1. The number of panelists should be kept to a minifnum and in line with 
the type O't program. 

2. Possibility of two-way video should be explored. 

3. More time should be allowed for interaction and question periods. Time 
limits for panelists responses should be set. 

4. Insert material from institutions should be better planneu. 

5. A better cueing system, for people who wish to talk. 

6. Panelists name should be flashed on screen iriore frequently for 
identification. 

7. Programme, may be lengthened. 

8. Sound quality in certain instances rL.uired improvement, e.g. volume 
and clarity. 

More lead time for program preparation and planning should be given. 
1'J. *3etter teamork and communication, at , the management studio level may be 
required. 

11. Programs should be better paced. 

1?. Pre-programmi ng acquaintance for panelists. 

Wrap up discussion would also be advisable for satisfaction of the 

participant. 

14. Physical layout of some on-site loca-tions could be improved. 
Vi. T.V. moni tors -juay need to be placed in better posi'tions. 

16. Ways should be explored to assure confidentiality on teleconferencing. 

PR06RAf>1 CONTENT 

17. Needs study required to determine program content. 

18. Content of programs should be geared to local audiences. 

19. Outline of program objectives and directions at beginning of sessions 
is required. 

20. Identi^f ication of types of graphics required for conveying information 
to viewers. 

21. Question preparation period should be set up to facilitate question 
period and responses.' 
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22. Panelists should have a broader range of v^iewpoints or expertise on 
subject rr.atter. 

23. More visual aides, e.g. slides, graphs, etc, should be utilized during 
1 ectures . 

24. Co-operation from higher administrative levels should be encouraged, 
■25. Consortium meinbers producing prograris should adequately budget for 

thei r programs . 

26. 'Program content should be presented in a straight forward, non confusing 

manner. ' 

COMMUNICATION 

27. A pre-air panel interaction session might be considered. 

28. Show less of typical T,V. talk shrw fornat and more informal two-v;ay 
audio conmunication. 

29. - Questions and answers should be clearly stated and interesting. 

30. Technical advisors should be on site to cope with technical problems. 

31. Explore possibilities of split screen two-way video. 

32. Anirr.ators may wish to infom and prepare themselves more adequately 
regarding the content of programs. 

33. Explore ways to have handset sockets or similar equipment to connect 
both circuits of an incoming telephone call directly into the prograrr 
feed during interactive period without having to use microphone and 
button. 

34. Protocol considerations as to order of questions/sites etc. should be 
conmunicated early in the programs. 

35. More interaction beLween sites should be encouraged. 

36. Increased pujlicity to encourage public participation is required. 

37. Animators shoiild explain procedures and equipment use more clearly. 

AUDIENCE 

38. Audience needs to be identi'^'ied and programs suited to audience and 
communi ty . 

39. More opinions rather than questions may be solicited. 

40. More audience participation might be encouraged. 

4:. Let audier.ce know what degree of participation/interaction they can 
anticipate. 
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Obtain audience size and I.D. inforjTia tion before- cormencing programs. 
Some wcrn>-up time at on site locations shojld be allowed for pri.or to 
satellite coninunication- This might help build up spontaneity c'jring 
irtteractive segment of the program. 
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APPENDIX D 



OBJECTIVES 
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To test the "utility* node! of a centrelized DE agency to 
co-ordinate cor^sortium activities. 

To test the concept and technical feasibility of interactive 
connunications systems, whereby st'jdents at a distance can 
ccrriiDunicate with a centrally located insiructor. 

To test the feasibility of various educational agencies producing 
and sharing DE programs for general or specific audiences, in 
a variety of modes (workshops, lectures, open lOrums, phone-ins, 
etc. ). 

To utilize S.'^'.E.P. as part of DEPG's planning process, in order 
to test the concept of using a range of educational institutions 
in a consortium model as the delivery system for distance 
educati on. 

To test the feasibility of using satellites as a DE mode, both 
technically and from an instructional perspective. 

To raise the level of awareness about distance education (what 
it is), DE techniques (how it is done}_jand the nature of the . 
demand for DE (v/ho benefits) on a province-wide basis. - 

To increase the community's awareness of the role .and potential 
of the regional community college. 

To test the various configurations of DE satellite-based syste..Ms, 
including hardware apsect? (cablevision , cl assrooi-is , telephone 
links etc.) and audience aspects (size and spatial distribution 
of groups, etc.). 

To test the feasibility fo satellite transmission and delivery 
at the specific site chosen for the S.T.E.P. experiment. 

To test the potential use of satellites as a cost effective method 
of transmitting live television signals and educational information, 
including data, on a province-wide basis, relative to alternative ^ 
systems (micro-wave systems, land lines, etc.). 



With program constraints, to assess whether the S.T.E.P. programming, 
ajs^.produced by the participating educational agencies, met a 
perceived need in the test communities. 
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A p p E r; D I X E 

RESULTS OF UTILITY ASSESSMENT OF RECOMENDATI 
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An arbitrary division was utilized to assess those recorrrnendat ions that 
had high utility 1000 - 666, medium utility 655 - 334 and low utility 333 - 0. 
It is possible to have the same recorrrnendation occupying different utility 
values because it may have utility on more thwn one objective. 

Recommendation Numbers Uti 1 i ty 
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